Minimizing regret in repeated play of games

These notes are largely based on a survey on learning, regret minimization and equilibria written by Avrim Blum and Yishay Mansour, and appearing as Chapter 4 of [NRTV07]. The main difference are in the presentation rather than in the content, and in the inclusion of Section 1.5. The notes correspond to two 2-hour lectures in the course on algorithmic game theory given in the Weizmann Institute in May 2013. Suppose player P is faced with a game for which P knows his own payoff matrix, but does not know the payoff matrix of the other players (or equivalently, knows payoff matrices for the other players, but has no reason to assume that other players are rational). In this general setting, one may represent (essentially without loss of generality) all remaining players as one player Q, and hence the game may be assumed to be a 2-player game. How should P play? If P has a dominant strategy, then playing it is a natural policy. But in the absence of dominant strategies, solution concepts such as Nash equilibrium involve the payoffs of both players, and hence P would not know which strategy corresponds to such a solution concept. Moreover, it might be the case that Q is not willing to play some strategies available to Q (e.g., they might be dominated by other strategies), so it could be that the payoff matrix for P (say, as a row player) contains columns that are not really part of the game. At this full generality of this situation, it is difficult to provide good advice for P what to play. The content of these notes is a variation on the above situation, in which some basic game G (for which P knows only his own payoff matrix) is repeated for T rounds, and P wishes to maximizes the sum of expected payoffs from all rounds. In this situation, P does have hope of playing intelligently, because at any given round P knows what Q played in previous rounds, and P can use this information in deciding what to play in the given round.